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Foreword 


The  Navy-wide  Personnel  Survey  (NPS)  is  the  major  product  of  the  Navy-wide 
Survey  Program  (NWSP).  Administered  regularly  since  1990  by  the  Navy  Personnel 
Research,  Studies,  and  Technology  (NPRST)  Division  of  the  Bureau  of  Naval  Personnel 
(BUPERS),  the  NPS  focuses  on  such  topics  as:  Sailor  demographics,  leadership 
satisfaction,  organizational  commitment,  performance  evaluations,  morale,  detailing, 
assignments,  job  characteristics,  job  satisfaction,  career  development,  availability  of 
resources,  gender  integration,  and  overall  satisfaction  with  Navy  life.  This  information 
provides  valuable  current  Navy  personnel  policies  and  programs.  The  NPS  is  conducted 
under  the  sponsorship  of  the  Chief  of  Naval  Personnel  (Ni)  within  the  NWSP  funding 
line.  That  funding  line  is  managed  by  the  Ni  Modeling  and  Analysis  Branch  (N104)  of 
BUPERS.  Because  of  the  critical  role  NPS  has  in  other  Navy  surveys  and  polls,  the  data 
were  analyzed  in  detail  using  advanced  statistical  techniques  to  develop  reliable  and 
valid  scales  and  indexes.  The  current  report  describes  the  development  and  validation  of 
short  forms  of  the  longer  NPS  scales.  Any  questions  regarding  this  report  should  be 
directed  to  the  NPS  Project  Director,  Dr.  Kimberly  Whittam, 
kimberly.whittam@navy.mil,  (901)  874-2321  or  DSN  882-2321. 


DAVID  L.  ALDERTON,  Ph.D. 

Director 


v 


Executive  Summary 


The  Navy-wide  Personnel  Survey  (NPS)  is  one  of  the  most  comprehensive  sources  of 
data  concerning  Navy  Sailors’  perceptions  of  the  quality  of  Navy  work  life  and  how  it 
impacts  them,  their  families,  and  the  Navy  organization  as  a  whole.  With  rich  data 
sources,  such  as  the  NPS,  a  common  challenge  is  developing  methods  for  combining  and 
presenting  the  data,  so  that  the  results  can  be  easily  disseminated  and  effectively  used. 
An  earlier  report  described  how  items  on  the  NPS  may  be  combined  into  19 
psychometric  scales,  representing  the  following  constructs:  (1)  availability  of  resources, 
(2)  morale,  (3)  gender  integration,  (4)  workplace  climate,  (5)  tempo,  (6)  impact  on 
personal  life,  (7)  immediate  supervisor,  (8)  command  leadership,  (9)  communication, 
(10)  job  security,  (11)  fairness,  (12)  Navy  image,  (13)  organizational  commitment,  (14) 
advancement/promotion,  (15)  performance  evaluations/fitness  reports,  (16) 
recognition,  (17)  career  progression,  (18)  career  counseling/guidance,  and  (19)  detailing 
(Bann,  Whittam,  &  Bar  nett- Walker,  2006).  Each  of  these  scales  demonstrated  good 
psychometric  properties. 

In  addition  to  developing  a  set  of  reliable  and  valid  individual  scales,  an  overall 
measure  of  Navy  Climate  was  developed.  The  Navy  Climate  Index  (NCI)  is  designed  to 
be  a  single  metric  that  captures  the  Navy’s  overall  “temperature”  or  “climate”  both 
currently  and  with  a  view  towards  the  future.  The  NCI  combined  the  following  seven 
scales  which  represented  indicators  influencing  Navy  Climate:  (1)  workplace  climate,  (2) 
organizational  commitment,  (3)  morale,  (4)  job  security,  (5)  communication,  (6) 
fairness,  and  (7)  Navy  image.  Scores  on  the  NCI  were  significantly,  positively  related  to 
several  outcomes,  including  perceptions  of  Navy  tone,  job  satisfaction,  and  retention 
intentions. 

Earlier  results  suggested  that  the  individual  construct  scales  and  the  Navy  Climate 
Index  provide  useful,  reliable,  and  valid  measures  for  assessing  Sailors’  perceptions  of 
the  Navy  experience.  However,  the  19  individual  scales  are  comprised  of  95  items. 
Administering  a  large  number  of  items  can  reduce  potential  measurement  error  of  the 
scales,  resulting  in  greater  reliability  and  validity.  However,  in  some  cases,  shorter  scales 
may  provide  important  practical  benefits  that  may  outweigh  a  loss  in  reliability  or 
validity,  including  reduced  time  and  costs  for  survey  administration,  lower  respondent 
burden,  and  improved  response  rates. 

This  report  briefly  outlines  our  exploration  of  potential  short  forms  for  the  19  scales 
on  the  2005  Navy-wide  Personnel  Survey.  Our  primary  goal  was  to  develop  l-item 
measures  for  each  of  the  scales  which  could  be  administered  when  time  and/or 
respondent  burden  is  of  particular  concern.  Using  both  statistical  and  content-based 
criteria,  we  successfully  developed  14  l-item  short  forms.  Short  forms  for  four  of  the 
remaining  scales  contained  two  items  each  and  the  short  form  for  the  final  scale 
contained  five  items,  primarily  due  to  concerns  about  ensuring  content  coverage.  Each 
of  the  short  forms  accounted  for  at  least  70  percent  of  the  variance  in  the  long  form 
scores  and  contained  items  with  high  levels  of  discrimination,  no  floor  or  ceiling  effects, 
and  appropriate  content. 
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In  addition  to  developing  short  forms  for  each  of  the  individual  scales,  a  short  form 
for  the  Navy  Climate  Index  was  also  developed,  referred  to  as  the  NCI-SF.  Confirmatory 
factor  analyses  supported  combining  the  short  forms  of  seven  scales  (Workplace 
Climate,  Organizational  Commitment,  Morale,  Job  Security,  Communication,  Fairness, 
and  Navy  Image)  into  a  single  NCI-SF  score.  The  NCI-SF  demonstrated  similar 
relationships  with  other  variables  (i.e.,  demographics,  tone,  job  satisfaction,  and 
retention  intentions)  as  the  NCI  long  form,  supporting  its  validity.  Furthermore,  the 
NCI-SF  was  strongly,  positively  correlated  with  the  NCI  (r  =  0.97)  and  accounted  for  94 
percent  of  the  variance  in  the  NCI  scores. 

In  conclusion,  the  results  suggest  that  several  of  the  scales  on  the  NPS  may  be 
substantially  shortened,  in  some  cases,  with  minimal  loss  of  reliability.  In  particular,  the 
findings  suggest  that  reducing  the  length  of  the  Navy  Climate  Index  by  73  percent  (i.e., 
removing  35  of  the  48  items  on  the  NCI)  produces  nearly  identical  scores,  given  the  high 
correlation  between  the  long  and  short  forms  of  the  NCI.  The  new  13-item  NCI-SF  that 
was  developed  could  be  administered  as  a  part  of  a  Navy  Quick  Poll  to  provide  Navy 
leadership  with  rapid  feedback  concerning  Navy  climate.  Future  studies  should  explore 
the  development  of  appropriate  cut-points  or  benchmarks  for  the  NCI-SF  in  relation  to 
important  outcomes,  such  as  retention  intentions. 
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Background 


The  Navy-wide  Personnel  Survey  (NPS)  is  one  of  the  most  comprehensive  sources  of 
data  concerning  Navy  Sailors’  perceptions  of  the  quality  of  Navy  work  life  and  how  it 
impacts  them,  their  families,  and  the  Navy  organization  as  a  whole.  With  rich  data 
sources,  such  as  the  NPS,  a  common  challenge  is  developing  methods  for  combining  and 
presenting  the  data,  so  that  the  results  can  be  easily  disseminated  and  effectively  used. 
An  earlier  report  described  how  items  on  the  NPS  may  be  combined  into  19 
psychometric  scales,  representing  the  following  constructs:  (1)  availability  of  resources, 
(2)  morale,  (3)  gender  integration,  (4)  workplace  climate,  (5)  tempo,  (6)  impact  on 
personal  life,  (7)  immediate  supervisor,  (8)  command  leadership,  (9)  communication, 
(10)  job  security,  (11)  fairness,  (12)  Navy  image,  (13)  organizational  commitment,  (14) 
advancement/promotion,  (15)  performance  evaluations/fitness  reports,  (16) 
recognition,  (17)  career  progression,  (18)  career  counseling/guidance,  and  (19)  detailing 
(Bann,  Whittam,  &  Bar  nett- Walker,  2006).  Each  of  these  scales  demonstrated  good 
psychometric  properties. 

In  addition  to  developing  a  set  of  reliable  and  valid  individual  scales,  an  overall 
measure  of  Navy  Climate  was  developed.  The  Navy  Climate  Index  (NCI)  is  designed  to 
be  a  single  metric  that  captures  the  Navy’s  overall  “temperature”  or  “climate”  both 
currently  and  with  a  view  towards  the  future.  The  NCI  combined  the  following  seven 
scales  which  represented  indicators  influencing  Navy  Climate:  (1)  workplace  climate,  (2) 
organizational  commitment,  (3)  morale,  (4)  job  security,  (5)  communication,  (6) 
fairness,  and  (7)  Navy  image.  Scores  on  the  NCI  were  significantly,  positively  related  to 
several  outcomes,  including  perceptions  of  Navy  tone,  job  satisfaction,  and  retention 
intentions. 

The  results  suggested  that  the  individual  construct  scales  and  the  Navy  Climate 
Index  provide  useful,  reliable,  and  valid  measures  for  assessing  Sailors’  perceptions  of 
the  Navy  experience.  However,  these  scales  require  the  administration  of  a  large 
number  of  items.  The  19  individual  scales  are  comprised  of  95  items.  Reducing  the 
survey  to  only  the  scales  comprising  the  Navy  Climate  Index  would  still  require  the 
administration  of  48  items.  Administering  a  large  number  of  items  can  reduce  potential 
measurement  error  of  the  scales,  resulting  in  greater  reliability  and  validity.  However,  in 
some  cases,  shorter  scales  may  provide  important  practical  benefits  that  may  outweigh  a 
loss  in  reliability  or  validity.  Shorter  scales  reduce  the  time  and  costs  of  survey 
administration.  They  also  lower  respondent  burden  and  can  be  particularly  useful  in 
situations,  such  as  in  times  of  high  operational  activity,  when  the  time  for  completing  a 
survey  may  be  limited.  Furthermore,  several  studies  have  found  an  inverse  relationship 
between  survey  length  and  response  rates  with  shorter  surveys  having  higher  response 
rates  (e.g.,  Kalantar  &  Talley,  1999;  McCarty,  House,  Harman,  &  Richards,  1999;  Steele, 
Schwendig,  &  Kilpatrick,  1992;  Tomaskovic-Devey,  Leiter,  &  Thompson,  1994).  Finally, 
even  if  the  length  of  a  survey  were  to  remain  the  same,  the  use  of  shorter  scales  would 
allow  other  scales  to  be  added  to  the  survey,  enabling  the  assessment  of  more  constructs 
within  the  same  time  period. 
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This  study  explored  reducing  the  length  of  the  19  NPS  scales  with  the  goals  of 
lowering  respondent  burden,  reducing  survey  administration  time  and  costs,  and 
increasing  response  rates..  The  primary  focus  was  on  assessing  the  feasibility  of  using 
single-item  scales.  Both  statistical  and  content  criteria  were  considered  when  developing 
the  short  forms  of  the  scales.  After  reducing  the  size  of  the  individual  scales,  a  short 
form  of  the  Navy  Climate  Index  was  developed  which  may  be  used  for  a  rapid 
assessment  of  the  current  Navy  climate. 


Scale  Short  Forms 


This  provides  an  overview  of  the  development  and  psychometric  evaluation  of  short 
forms  for  the  19  scales  included  on  the  2005  Navy-wide  Personnel  Survey.  The  2005 
NPS  was  administered  on  the  Internet  between  March  20  and  June  20,  2005  to  a 
stratified  random  sample  of  16,372  active  duty  Sailors.  There  were  a  total  of  3,610  usable 
surveys  which,  after  adjusting  for  incomplete  surveys  and  non-contacts,  resulted  in  an 
overall  response  rate  of  26  percent  (unweighted)  and  37  percent  (weighted  to  the 
population). 

To  develop  the  short  forms,  researchers  examined  several  psychometric  properties  of 
the  individual  items,  including  their  means,  standard  deviations,  and  item  response 
theory  (IRT)  parameters,  and  assessed  the  relationship  between  the  short  and  long 
forms  of  the  scales,  using  ordinary  least  squares  regression.  Before  beginning  the  short 
form  development  and  evaluation  process,  the  following  criteria  was  established  for  a 
short  form  scale  to  be  considered  an  acceptable  substitute  for  the  long  form  scale: 

Ceiling  and/or  Floor  Effects.  To  avoid  potential  ceiling  or  floor  effects,  the  items 
selected  for  the  short  forms  should  not  have  means  that  are  near  the  minimum  or 
maximum  possible  scores. 

Variability  in  Item  Responses.  The  standard  deviations  for  each  item  as  a 
measure  of  the  variability  in  item  responses  were  examined.  Items  for  which  all 
respondents  respond  similarly  are  not  informative  and  should  not  be  included  on  the 
short  form. 

Item  Response  Theory  Parameters.  In  an  earlier  report,  IRT  parameters  were 
computed  for  each  item  using  the  graded  response  model  (Samejima,  1969).  Items 
comprising  each  short  form  must  have  an  IRT  slope  of  one  or  greater,  indicating  good 
discrimination,  and  a  spread  of  IRT  threshold  parameters. 

Variability  in  Long  Form  Scores.  Each  short  form  was  regressed  on  the  long 
form  scores  to  determine  the  percentage  of  variability  in  the  long  form  scores  accounted 
for  by  the  short  form  scores.  Scores  on  the  short  form  were  required  to  account  for  at 
least  70  percent  of  the  variance  in  the  long  form  scores  to  be  an  acceptable  short  form. 

Content  Considerations.  In  addition  to  the  statistical  properties  of  the  items,  the 
content  of  the  items  was  evaluated  when  developing  the  short  forms.  Overall, 
researchers  sought  to  select  items  that  had  general,  rather  than  specific,  content. 

General  items  capture  participants’  broader  perspectives  of  a  construct  rather  than 
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restricting  them  to  specific  elements  of  the  construct.  For  example,  all  else  being  equal, 
item  23F  (satisfied  with  immediate  supervisor),  a  general  item  on  the  Immediate 
Supervisor  scale,  should  more  effectively  capture  participants’  overall  perceptions  of 
their  immediate  supervisor  than  a  more  specific  item,  such  as  item  23A  (adequate 
training/expertise),  which  would  assess  only  the  participant’s  perceptions  of  his/her 
supervisor’s  training  and  could  miss  other  important  aspects  of  the  supervisor’s 
performance,  such  as  his/her  responsiveness  to  concerns  and  interactions  with  others. 

Ideally,  the  scales  could  be  reduced  to  the  shortest  scale  possible,  namely  a  l-item 
scale.  Research  began  by  testing  individual  items  to  determine  their  suitability  as  l-item 
scales.  However,  if  none  of  the  items  were  considered  adequate  as  a  one-item  scale, 
based  on  the  criteria  outlined  above,  a  scale  with  two  or  more  items  was  then  developed. 
The  following  sections  described  the  short  form  development  and  evaluation  for  each  of 
the  19  scales. 


Availability  of  Resources 

To  develop  a  short  form  for  the  Availability  of  Resources  scale,  researchers  began  by 
examining  the  item  descriptive  statistics  shown  in  Table  1.  Overall,  the  four  items  on  the 
scale  demonstrate  good  psychometric  properties  (i.e.,  no  ceiling  or  floor  effects,  high 
IRT  slopes,  and  spread  of  IRT  threshold  parameters).  However,  none  of  the  items 
individually  accounted  for  at  least  70  percent  of  the  variance  in  the  long  form  scores  and 
therefore,  would  not  be  suitable  as  l-item  scales. 

When  reviewing  the  content  of  the  items,  it  appears  that  they  split  into  two  general 
content  areas:  (1)  people-oriented  resources  (item  8A  [adequate  qualified  personnel] 
and  item  8D  [adequate  Navy  support  services]),  and  (2)  equipment-related  resources 
(item  8B  [adequate  tools]  and  item  8C  [adequate  spare  parts  and/or  supplies]).  To 
ensure  that  both  content  areas  were  covered,  a  2-item  short  form  was  developed, 
containing  one  item  from  each  content  area.  Item  8 A  (adequate  qualified  personnel) 
was  selected  to  represent  people-oriented  resources  because  it  has  a  higher  threshold 
parameter  and  accounts  for  a  greater  percentage  of  variance  in  long  form  scores  than 
item  8D  (adequate  Navy  support  services).  While  item  8B  (adequate  tools)  has  a  higher 
threshold  parameter  and  accounts  for  slightly  more  variance  then  item  8C  (adequate 
spare  parts  and/or  supplies),  item  8C  also  performs  well  and  was  selected  for  inclusion 
in  the  short  form,  given  prior  results  from  the  NPS  and  anecdotal  evidence  which 
suggests  spare  parts  are  a  particularly  salient  concern  among  Sailors. 

Scores  for  the  resulting  2-item  short  form  were  computed,  using  the  mean  of 
responses  to  items  8A  and  8C.  The  short  form  accounts  for  84  percent  of  the  variance  in 
the  Availability  of  Resources  long  form  scores.  To  graphically  represent  the  relationship 
between  the  short  and  long  forms,  the  mean  long  form  scores  were  plotted  according  to 
five  categories  based  on  mean  responses  to  the  short  form.  The  mean  short  forms  were 
placed  into  categories  as  follows:  (1)  strongly  disagree  (mean  of  0.0-0. 9),  (2)  disagree 
(mean  of  1.0-1.9),  (3)  neither  agree  nor  disagree  (mean  of  2. 0-2.9),  (4)  agree  (mean  of 
3.0-3. 9),  and  (5)  strongly  agree  (mean  of  4. 0-5.0).  As  shown  in  Figure  1,  there  is  a 
strong,  positive  relationship  between  the  two  forms  with  mean  long  form  scores 
increasing  as  mean  scores  on  the  short  form  increase. 
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Table  1 

Availability  of  Resources  scale 


Item 

%  Scale 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q8A 

Adequate  qualified  personnel 

57.1 

2.15 

1.02 

1.96 

-2.45 

-1.45 

-0.94 

0.84 

Q8B 

Adequate  tools 

69.4 

2.29 

1.02 

4.58 

-1.89 

-1.05 

-0.59 

0.90 

Q8C 

Adequate  spare  parts  and/or 
supplies 

64.2 

2.63 

1.11 

2.28 

-1.87 

-0.90 

-0.18 

1.43 

Q8D 

Adequate  Navy  support 
services 

46.6 

2.29 

1.03 

1.25 

-2.91 

-1.76 

-0.91 

1.43 
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Categories  of  Mean  Availability  of  Resources-Short  Form  Scores 


Figure  1.  Mean  Availability  of  Resources  scale  scores  by  mean  short  form 

scores. 


Morale 

The  descriptive  statistics  for  the  16  items  comprising  the  Morale  scale  are  displayed 
in  Table  2.  As  shown  in  the  table,  none  of  the  items  accounts  for  at  least  70  percent  of 
the  variance  in  the  Morale  scale  scores.  In  addition,  reviewing  the  content  of  the  items 
suggests  that  they  may  be  measuring  multiple  aspects  of  Morale,  including  performance 
evaluations  (items  10A,  10B,  and  10P),  supplies  (10C),  training  (items  10D  and  10E),  co¬ 
workers  and  leaders  (items  10F-H,  10K,  and  10N),  and  workload  and  compensation 
(items  101, 10J,  10L,  10M,  and  10O).  Using  a  l-item  scale  may  not  capture  all  of  these 
diverse  factors.  Therefore,  one  item  from  each  of  the  five  constructs  was  included  to 
form  a  short  form.  Based  on  content,  percentage  of  variance  accounted  for,  and  the  item 
parameters,  the  following  five  items  were  chosen  for  inclusion  in  the  Morale  short  form: 

•  Item  10B  (Performance  evaluation  system), 

•  Item  10C  (Supply  of  spare  parts/supplies), 

•  Item  10D  (Quality  of  Navy  training  programs), 

•  Item  10 H  (Command  leadership),  and 

•  Item  10J  (Workload). 

Similar  to  the  Availability  of  Resources  scale,  scores  for  the  Morale  short  form  were 
computed  as  the  mean  of  the  items  comprising  the  scale.  The  Morale  short  form 
accounted  for  84  percent  of  the  variance  in  the  long  form  scores.  As  shown  in  Figure  2, 
there  is  a  strong,  positive,  linear  relationship  between  the  short  and  long  forms. 
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Table  2 
Morale  scale 


Item 

# 

Description 

%  Scale 
Variance 

Mean 

SD 

a 

1 RT  Parameters 
bl  b2  b3 

b4 

Q10A 

Advancement/  promotion 
opportunities 

31.8 

2.61 

1.06 

1.20 

-2.90 

-1.22 

-0.25 

2.16 

Q10B 

Performance  evaluation 
system 

39.8 

2.66 

0.99 

1.50 

-2.59 

-1.11 

-0.06 

2.24 

Q10C 

Supply  of  spare  parts/supplies 

32.3 

2.98 

0.93 

1.27 

-2.69 

-0.93 

0.82 

3.14 

Q10D 

Quality  of  Navy  training 
programs 

41.5 

2.57 

0.90 

1.64 

-2.65 

-1.41 

-0.12 

2.21 

QlOE 

Quality  of  education  programs 

36.6 

2.33 

0.90 

1.48 

-2.97 

-1.84 

-0.54 

1.72 

QlOF 

Co-  workers/  sh  i  pmates 

37.7 

2.26 

1.00 

1.47 

-2.88 

-1.51 

-0.79 

1.35 

QIOG 

1  mmediate  supervisor 

39.9 

2.33 

1.08 

1.57 

-2.36 

-1.34 

-0.62 

1.26 

Q10H 

Command  leadership 

48.9 

2.51 

1.19 

1.89 

-1.83 

-0.91 

-0.32 

1.21 

QIOI 

Pace  of  work 

50.6 

2.82 

1.08 

2.34 

-1.68 

-0.65 

0.08 

1.87 

QIOJ 

Workload 

51.0 

2.96 

1.12 

2.27 

-1.50 

-0.50 

0.26 

1.92 

QlOK 

Unit/workgroup  manning 

44.9 

2.98 

1.10 

1.87 

-1.66 

-0.53 

0.33 

2.21 

QIOL 

Pay/  bonuses/ other 
compensation 

30.8 

2.63 

1.01 

1.16 

-2.74 

-1.59 

0.04 

2.29 

QIOM 

Amount  of  time  off 

46.5 

2.66 

1.20 

1.77 

-1.69 

-0.81 

-0.17 

1.50 

QION 

Navy  support  services 

26.9 

2.56 

0.97 

1.17 

-2.95 

-1.72 

-0.16 

2.36 

QlOO 

TEMPO 

33.3 

3.10 

1.07 

1.30 

-1.87 

-0.72 

0.97 

2.57 

QlOP 

Performance  of  crew,  work 
team  on  exercises 

34.9 

2.36 

0.85 

1.43 

-3.22 

-2.04 

-0.42 

1.88 

Categories  of  Mean  Morale-Short  Form  Scale  Scores 


Figure  2.  Mean  Morale  scale  scores  by  mean  short  form  scores. 


Gender  I  ntegration 

As  shown  in  Table  3,  items  12B  (women  have  ability  to  carry  out  combatant  roles) 
and  12C  (women  are  being  successfully  integrated)  each  individually  account  for 
approximately  72  percent  of  the  variance  in  the  Gender  Integration  scale.  Both  have 
high  slope  parameters,  mid-range  mean  values,  and  variability  in  item  responses.  Item 
12C  was  selected  for  use  as  the  l-item  Gender  Integration  scale  due  to  focus  on  gender 
integration  in  general  rather  than  specifically  in  combat  roles.  Figure  3  plots  the  mean 
Gender  Integration  scale  scores  by  responses  to  item  12C.  The  strong,  linear  relationship 
in  the  graph  supports  the  equivalence  of  the  l-item  short  form  and  the  Gender 
Integration  scale  long  form. 
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Table  3 

Gender  I  ntegration  scale 


Item 

Description 

% 

1 RT  Parameters 

# 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q12A 

Supportive  leadership 

63.3 

1.90 

0.90 

1.78 

-2.91 

-2.20 

-1.19 

0.46 

Q12B 

Women  have  ability  to  carry 
out  combatant  roles 

72.5 

2.05 

0.98 

3.38 

-2.03 

-1.51 

-0.76 

0.52 

Q12C 

Women  are  being  successfully 
integrated 

72.4 

2.21 

0.96 

2.82 

-2.13 

-1.54 

-0.58 

0.84 

00 


Item  12C  (Women  are  being  successfully  integrated  into  combatant  ships 
and  aviation  squadrons) 


Figure  3.  Mean  Gender  I  ntegration  scale  scores  by  1-item  gender  integration 

measure. 


Workplace  Climate 

The  item-level  descriptive  statistics  for  the  Workplace  Climate  scale  are  presented  in 
Table  4.  None  of  the  items  on  the  scale  accounts  for  at  least  70  percent  of  the  variance  in 
the  long  form  scores  by  itself;  however,  item  13D  (opportunity  for  personal  growth  and 
development  on  the  job)  accounts  for  66  percent  of  the  scale  variance.  To  construct  a  2- 
item  short  form,  item  13D  was  selected  first  based  on  the  large  percentage  of  variance 
for  which  it  accounts  and  its  large  IRT  slope  (a)  parameter.  The  remaining  items  were 
then  examined  with  large  slope  parameters  and  selected  item  13B  (amount  of 
responsibility  I  have  in  my  job)  for  the  second  item  on  the  scale.  This  2-item  short  form 
demonstrated  a  strong  relationship  with  the  long  form  scores  (see  Figure  4)  and 
accounted  for  80  percent  of  the  variance  in  the  long  form  scores. 
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Table  4 

Workplace  Climate  scale 


Item 

# 

Description 

% 

Variance 

Mean 

SD 

a 

1 RT  Parameters 
bl  b2  b3 

b4 

Q13A 

Amount  of  freedom  1  am  given 
to  do  my  job 

55.9 

2.11 

1.03 

2.19 

-2.36 

-1.29 

-0.77 

0.70 

Q13B 

Amount  of  responsibility  1  have 
in  my  job 

61.4 

2.02 

0.96 

3.28 

-2.18 

-1.30 

-0.77 

0.57 

Q13C 

Amount  of  challenge  in  my  job 

61.7 

2.15 

1.04 

3.37 

-1.91 

-1.13 

-0.59 

0.66 

Q13D 

Opportunity  for  personal  growth 
and  development  on  the  job 

65.8 

2.37 

1.13 

2.97 

-1.76 

-0.91 

-0.38 

0.87 

Q13E 

Feeling  of  accomplishment  1  get 
from  doing  my  job 

62.1 

2.26 

1.13 

2.87 

-1.76 

-1.01 

-0.47 

0.74 

Q13F 

J  ob  security 

36.4 

2.00 

0.98 

1.36 

-3.07 

-2.15 

-1.09 

0.68 

Q13G 

Physical  working  conditions  of 
my  work  site 

36.5 

2.17 

0.98 

1.37 

-3.12 

-1.81 

-0.94 

1.16 

Q13I 

Flexibility  in  dealing  with 
family/ personal  issues 

40.8 

2.14 

1.10 

1.26 

-2.70 

-1.81 

-0.83 

0.82 

1  H  I  I  I  I  I 

Very  dissatisfied  Dissatisfied  Neither  satisfied  Satisfied  Very  satisfied 

nor  dissatisfied 


Categories  of  Mean  Workplace  Climate-Short  Form  Scale  Scores 


Figure  4.  Mean  Workplace  Climate  scale  scores  by  mean  short  form  scores. 
TEMPO 

All  three  items  on  the  TEMPO  Scale  appeared  to  have  good  psychometric  properties 
although  two  of  the  items  (item  19A  and  19B)  had  higher  slope  parameters  (a  =  3.12  and 
3.13,  respectively)  than  item  19C  (see  Table  5).  These  two  items  also  accounted  for 
approximately  72  percent  of  the  variance  in  the  long  form  parameters.  Item  19A  (time 
spent  at  permanent  duty  station)  was  selected  for  use  as  the  l-item  Tempo  short  form 
because  the  term  “permanent  duty  station”  is  more  general  than  the  term  “shore  duty” 
used  in  item  19B  (time  spent  on  shore  duty)  and  could  potentially  be  applied  to  a  variety 
of  personnel,  including  both  Sailors  and  civilian  personnel.  As  expected,  mean  Tempo 
scale  scores  increase  as  agreement  with  this  l-item  measure  increases  (see  Figure  5). 
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Table  5 
TEMPO  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q19A 

Time  spent  at  permanent  duty 
station 

72.2 

2.22 

1.08 

3.12 

-1.93 

-1.21 

-0.57 

0.66 

Q19B 

Time  spent  on  shore  duty 

72.6 

2.35 

1.10 

3.13 

-1.79 

-1.19 

-0.36 

0.80 

Q19C 

Time  spent  on  sea  duty 

57.4 

2.42 

1.02 

1.52 

-2.63 

-1.62 

-0.30 

1.32 

to 
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Strongly  Disagree  Neither  agree  Agree  Strongly  agree 

disagree  nor  disagree 

Q19A  (I  am  satisfied  with  the  amount  of  time  I  am  able  to  spend  at  my 
permanent  duty  station) 


Figure  5.  Mean  TEMPO  scale  scores  by  1-item  TEMPO  measure. 


I  mpact  on  Personal  Life 

Based  on  the  item  characteristics  presented  in  Table  6,  item  21A  (career  gets  in  way 
of  personal  life)  was  selected  for  use  as  a  l-item  Impact  on  Personal  Life  scale.  This  item 
had  the  highest  percentage  of  variance  (79%),  standard  deviation  (1.17),  and  IRT  slope 
parameter  (4.21).  It  also  has  general  content  and  demonstrated  a  strong  relationship 
with  the  Impact  on  Personal  Life  long  form  scores  (see  Figure  6). 
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Table  6 

I  mpact  on  Personal  Life  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q21A 

Career  gets  in  way  of  personal 
life 

78.5 

2.89 

1.17 

4.21 

-1.55 

-0.41 

0.26 

1.17 

Q21B 

Career  causes  significant 
separation  from  family 

67.9 

2.39 

1.13 

2.00 

-2.31 

-1.04 

-0.31 

0.92 

Q21C 

Difficulty  juggling  career  and 
personal  life 

71.5 

3.18 

1.14 

2.37 

-1.48 

-0.21 

0.65 

1.68 

Item  21A  (My  Navy  career  gets  in  the  way  of  my  ability  to  have  or  maintain 

a  personal  life) 


Figure  6.  Mean  I  mpact  on  Personal  Life  scale  scores  by  1-item  impact  on 

personal  life  measure. 


I  mmediate  Supervisor 

After  reviewing  the  item-level  descriptive  statistics  shown  in  Table  7,  item  23F 
(satisfied  with  immediate  supervisor)  was  selected  for  use  as  a  l-item  Immediate 
Supervisor  measure.  This  single  item  accounted  for  90  percent  of  the  variance  in  the 
long  form  scores.  It  also  had  the  largest  slope  parameter  (9.14);  standard  deviation 
(1.14),  a  spread  of  threshold  parameters,  and  a  strong,  linear  relationship  with  the  long 
form  scores  (see  Table  7  and  Figure  7). 
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Table  7 

I  mmediate  Supervisor  scale 


Item 

# 

Description 

% 

Variance 

Mean 

SD 

a 

1 RT  Parameters 
bl  b2  b3 

b4 

Q23A 

Adequate  training/ expertise 

66.8 

1.93 

0.99 

3.07 

-1.87 

-1.13 

-0.74 

0.54 

Q23B 

Deals  well  with 

subordinates 

84.6 

2.17 

1.14 

5.60 

-1.36 

-0.76 

-0.36 

0.65 

Q23C 

Deals  well  with  superiors 

70.5 

2.05 

0.99 

3.36 

-1.79 

-1.11 

-0.52 

0.71 

Q23D 

Provides  adequate  support 
and  guidance 

85.3 

2.23 

1.14 

5.59 

-1.34 

-0.74 

-0.27 

0.73 

Q23E 

Responsive  to  Sailor  needs 
and  concerns 

79.9 

2.15 

1.09 

4.89 

-1.41 

-0.86 

-0.38 

0.69 

Q23F 

Satisfied  with  immediate 
supervisor 

89.8 

2.14 

1.14 

9.14 

-1.24 

-0.75 

-0.38 

0.59 
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Strongly 

Disagree 

Neither  agree 

Agree 

Strongly  agree 

disagree 

nor  disagree 

Item  23F  (Overall,  I  am  satisfied  with  my  immediate  supervisor) 


Figure  7.  Mean  I  mmediate  Supervisor  scale  scores  by  1-item  immediate 

supervisor  measure. 


Command  Leadership 

Similar  to  the  Immediate  Supervisor  scale,  a  general  satisfaction  item,  item  24F 
(satisfied  with  command  leadership)  was  chosen  as  a  l-item  Command  Leadership 
measure.  This  item  had  the  highest  standard  deviation  and  discrimination,  accounted 
for  89  percent  of  the  variance  in  the  long  form  and  was  strongly  related  to  the  long  form 
scores  (see  Table  8  and  Figure  8). 
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Table  8 

Command  Leadership  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q24A 

Adequate  training/ expertise 

66.1 

1.96 

0.90 

2.95 

-0.71 

0.75 

1.35 

2.05 

Q24B 

Deals  well  with  subordinates 

84.5 

2.27 

1.07 

5.39 

-0.89 

0.32 

0.80 

1.47 

Q24C 

Deals  well  with  superiors 

70.7 

2.08 

0.90 

3.38 

-0.90 

0.49 

1.26 

1.94 

Q24D 

Provides  adequate  support 
and  guidance 

84.1 

2.31 

1.08 

5.63 

-0.94 

0.27 

0.80 

1.36 

Q24E 

Responsive  to  Sailor  needs 
and  concerns 

81.3 

2.27 

1.09 

5.13 

-0.88 

0.32 

0.83 

1.33 

Q24F 

Satisfied  with  command 
leadership 

88.7 

2.29 

1.10 

7.93 

-0.84 

0.30 

0.78 

1.28 

4.87 


Strongly 

disagree 


Disagree  Neither  agree  Agree  Strongly  agree 
nor  disagree 


Item  24F  (Overall,  I  am  satisfied  with  my  command  leadership) 


Figure  8.  Mean  Command  Leadership  scale  scores  by  1-item  command 

leadership  measure. 

Communication 

While  the  items  on  the  Communication  scale  met  the  criteria  for  the  means, 
standard  deviations,  and  IRT  parameters,  none  of  the  items  alone  accounted  for  at  least 
70  percent  of  the  variance  in  the  long  form  scores  (Table  9).  To  develop  a  2-item  short 
form,  researchers  first  selected  item  25D  (command  leadership  keeps  me  informed  of 
Navy  policies)  which  accounts  for  the  largest  percentage  of  variance  (66%)  and  has  the 
highest  discrimination  (i.e.,  IRT  slope  parameter).  Item  25A  (Navy  clearly 
communicates  goals  and  strategies)  was  then  selected  as  the  second  item  for  inclusion  in 
the  scale.  While  items  25B  (senior  leadership  keeps  Sailors  informed)  and  25C 
(command  leadership  communicates  positive  attitude  about  Navy)  have  higher  slope 
parameters  and  account  for  more  variance,  item  25A  was  selected  because  item  25B  has 
very  similar  content  to  item  25D  and  item  25C  is  too  specific  given  its  focus  on  positive 
attitude.  Combined  items  25A  and  25D  account  for  82  percent  of  the  variance  in  the 
scores  on  the  Communication  long  form  and  as  shown  in  Figure  9,  mean  scores  on  the 
Communication  long  form  increase  with  increasing  mean  scores  on  the  short  form. 
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Table  9 

Communication  scale 


Item 

# 

Description 

% 

Variance 

Q25A 

Navy  clearly  communicates 
goals  and  strategies 

50.6 

Q25B 

Senior  leadership  keeps  Sailors 
informed 

64.2 

Q25C 

Command  leadership 
communicates  positive 
attitude  about  Navy 

50.1 

Q25D 

Command  leadership  keeps  me 
informed  of  Navy  policies 

65.8 

Q25E 

Someone  in  chain  of  command 

talked  about  new  career 

initiatives 

57.3 

I RT  Parameters 


Mean 

SD 

a 

bl 

b2 

b3 

b4 

2.55 

1.00 

1.77 

-2.57 

-1.10 

-0.33 

1.82 

2.55 

1.01 

2.58 

-2.11 

-0.95 

-0.32 

1.58 

2.20 

0.95 

1.87 

-2.57 

-1.64 

-0.77 

1.08 

2.35 

0.97 

3.16 

-2.14 

-1.16 

-0.49 

1.14 

2.82 

1.25 

1.74 

-1.67 

-0.58 

-0.03 

1.53 

Categories  of  Mean  Communication-Short  Form  Scale  Scores 


Figure  9.  Mean  Communication  scale  scores  by  mean  short  form  scores. 

J  ob  Security 

After  reviewing  the  item  psychometric  properties  and  content  shown  in  Table  to, 
item  26C  (future  in  Navy  appears  secure)  was  selected  as  a  l-item  short  form  for  the  Job 
Security  scale.  The  item  has  content  related  to  general  job  security,  has  the  highest  slope 
parameter  (a  =  4.14),  accounts  for  the  largest  percentage  of  variance  in  Job  Security 
scale  scores  (78%),  and  shows  a  strong,  linear  relationship  with  the  Job  Security  scale 
(see  Figure  10). 
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Table  10 

J  ob  Security  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q26A 

Feel  positive  about  future 

Navy  career 

72.3 

2.58 

1.20 

2.24 

-1.74 

-0.84 

-0.21 

l.ll 

Q26B 

Navy  is  doing  all  it  can  to 
protect  my  job  security 

73.9 

2.76 

1.09 

2.84 

-1.67 

-0.77 

0.12 

1.42 

Q26C 

Future  in  Navy  appears  secure 

78.3 

2.39 

1.12 

4. 14 

-1.62 

-0.96 

-0.41 

0.87 

IO 

IO 


Item  26C  (My  future  in  the  Navy  appears  secure  as  long  as  I  do  a  good  job) 


Figure  10.  Mean  J  ob  Security  scale  scores  by  1-item  job  security  measure. 


Fairness 

Item  27D  (confident  policies  affecting  size  of  Navy  will  be  administered  fairly  and 
consistently)  was  selected  as  a  l-item  short  form  for  the  Fairness  scale.  Although  this 
item  accounted  for  slightly  less  than  70  percent  of  the  variance  in  long  form  scores,  it 
demonstrated  the  highest  level  of  discrimination  (a  =  2.85),  was  strongly,  positively 
related  to  long  form  scores,  and  conceptually  appeared  to  be  most  related  to  fairness 
(see  Table  11  and  Figure  11). 
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Table  11 
Fairness  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q27A 

Navy  personnel  policies  seem 
fair 

64.0 

2.59 

1.03 

2.27 

-1.99 

-1.01 

-0.21 

1.62 

Q27B 

Navy  policies  retain  best 
quality  Sailors 

71.6 

2.99 

1.17 

2.70 

-1.37 

-0.44 

0.24 

1.68 

Q27C 

T rust  Navy  to  look  out  for  my 
best  interests 

70.8 

3.20 

1.13 

2.73 

-1.20 

-0.28 

0.55 

1.95 

Q27D 

Confident  policies  affecting 
size  of  Navy  will  be 
administered  fairly  and 
consistently 

69.5 

2.92 

1.06 

2.85 

-1.50 

-0.65 

0.27 

1.87 

Item  27D  (I  am  confident  that  policies  that  affect  the  size  of  the  Navy  will  be 
administered  fairly  and  consistenty) 


Figure  11.  Mean  Fairness  scale  scores  by  1-item  fairness  measure. 


Navy  I  mage 

Item  28C  (I  would  recommend  the  Navy  as  a  good  place  to  work)  was  chosen  as  a  1- 
item  measure  of  Navy  Image.  This  item  met  all  of  the  criteria  specified  for  the  selection 
of  a  short  form;  it  accounted  for  76  percent  of  the  variance  in  Navy  Image  scale  scores, 
had  mid-range  mean  values,  variability  in  item  responses,  the  highest  IRT  slope 
parameter,  a  strong,  linear  relationship  with  the  long  form  scores,  and  general  item 
content  (see  Table  12  and  Figure  12). 
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Table  12 

Navy  I  mage  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q28A 

1  would  encourage  others  to  join 
the  Navy 

74.0 

2.47 

1.15 

5.69 

-1.29 

-0.72 

-0.22 

0.84 

Q28B 

1  talk  about  Navy  to  friends  as  a 
good  organization 

72.5 

2.25 

1.03 

5.37 

-1.57 

-0.99 

-0.42 

0.77 

Q28C 

1  would  recommend  the  Navy  as 
a  good  place  to  work 

76.3 

2.40 

1.08 

7.39 

-1.40 

-0.83 

-0.26 

0.84 

Q28D 

1  would  wear  civilian  clothing 
with  Navy  logos 

42.4 

2.63 

1.25 

1.39 

-1.83 

-0.89 

-0.24 

1.43 

Q28E 

1  nformation  1  hear  about  Navy 
from  non- Navy  sources  is 
usually  positive 

42.1 

2.57 

0.94 

1.42 

-2.72 

-1.38 

-0.15 

2.20 

Q28F 

Comments  1  hear  about  Navy 
from  fellow  Sailors  are  usually 
positive 

55.9 

3.07 

1.06 

1.88 

-1.63 

-0.40 

0.48 

2.46 

Q28G 

Navy  of  tomorrow  will  be  better 
than  today 

43.3 

2.86 

1.05 

1.35 

-2.09 

-1.06 

0.56 

2.19 

1  ^ - I - I - I - I - 1 

Strongly  Disagree  Neither  agree  Agree  Strongly  agree 

disagree  nor  disagree 

Item  28C  (I  would  recommend  the  Navy  as  a  good  place  to  work) 


Figure  12.  Mean  Navy  I  mage  scale  scores  by  1-item  Navy  image  measure. 


Organizational  Commitment 

Similar  to  the  Navy  Image  scale,  there  was  a  single  item  that  appeared  to  stand  out 
as  a  potential  l-item  short  form.  Item  37E  (I  feel  a  strong  sense  of  belonging  in  the 
Navy)  accounted  for  84  percent  of  the  variance  in  the  Organizational  Commitment  scale 
scores  and  met  the  other  criteria  for  selection  as  a  short  form  (see  Table  13  and  Figure 
13). 
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Table  13 

Organizational  Commitment  scale 


Item 

%  Scale 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q37A 

Navy  has  personal  meaning 
for  me 

72.7 

2.08 

1.05 

3.41 

-1.96 

-1.27 

-0.59 

0.51 

Q37B 

1  feel  like  I'm  "part  of  the 
family"  in  the  Navy 

78.7 

2.41 

1.16 

4.11 

-1.57 

-0.86 

-0.24 

0.78 

Q37C 

1  feel  "emotionally  attached" 
to  the  Navy 

83.7 

2.60 

1.23 

4.86 

-1.29 

-0.66 

-0.02 

0.87 

Q37D 

1  could  not  easily  become 
attached  to  another 
organization 

68.4 

2.85 

1.24 

2.52 

-1.38 

-0.55 

0.26 

1.18 

Q37E 

1  feel  a  strong  sense  of 
belonging  in  the  Navy 

84.3 

2.46 

1.17 

5.33 

-1.40 

-0.81 

-0.17 

0.81 

Item  37E  (I  feel  a  strong  sense  of  belonging  in  the  Navy) 


Figure  13.  Mean  Organizational  Commitment  scale  scores  by  1-item 
organizational  commitment  measure. 


Advancement/  Promotion 

Consistent  with  other  scales,  such  as  Immediate  Supervisor  and  Command 
Leadership,  which  contained  an  overall  satisfaction  item,  item  38B  (satisfied  with 
advancement/promotion  system)  was  selected  as  the  l-item  measure  for  the 
Advancement/Promotion  scale.  This  item  performed  best  of  the  three  items  on  the 
scale,  accounted  for  77  percent  of  the  variance  in  long  form  scores,  and  demonstrated 
the  pattern  of  scores  expected  of  a  suitable  short  form  scale  (see  Table  14  and  Figure  14). 
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Table  14 

Advancement/  Promotion  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q38A 

Clear  understanding  of 
advancement/  promotion 
system 

41.6 

1.98 

0.96 

0.95 

-4.29 

-2.62 

-1.74 

0.94 

Q38B 

Satisfied  with  advancement/ 
promotion  system 

76.7 

2.98 

1.26 

4.89 

-1.12 

-0.32 

0.16 

1.27 

Q38C 

The  most  qualified  Sailors  get 
promoted 

70.1 

3.23 

1.25 

2.13 

-1.15 

-0.09 

0.52 

1.78 

CO 

o 


5  i 


4.69 


o 

o 

C/) 

0 

0 

o 

CO 

c 

o 


c 

0 

E 

0 

O 

c 

0 

> 

TD 

< 


- 1 - 1 - 1 - 1 - 1 

Strongly  Disagree  Neither  agree  Agree  Strongly  agree 

disagree  nor  disagree 

Item  38B  (I  am  satisfied  with  the  present  Navy  advancement/promotion 

system) 


Figure  14.  Mean  Advancement/  Promotion  scale  scores  by  1-item 
advancement/  promotion  measure. 


Performance  Evaluations/  Fitness  Reports 

The  Performance  Evaluation/Fitness  Reports  scale  includes  an  overall  satisfaction 
item,  item  39F  (satisfied  with  present  EVAL/FITREP  system),  however,  this  item  only 
accounts  for  56  percent  of  the  variance  in  the  scale  scores  (see  Table  15).  Therefore,  we 
developed  a  2-item  short  form  which  includes  item  39F  along  with  an  additional  item 
from  the  scale,  item  39B  (Last  EVAL/FITREP  was  fair/accurate).  Item  39B  was  selected 
due  to  its  high  IRT  slope  parameter  and  percentage  of  variance.  Together  these  two 
items  accounted  for  81  percent  of  the  variance  in  long  forms  and  increases  in  mean 
scores  on  the  2-item  short  form  are  associated  with  increases  in  mean  scores  on  the  long 
form  (see  Figure  15). 


31 


Table  15 

Performance  Evaluations/  Fitness  Reports  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q39A 

Clear  understanding  of  present 
system 

35.2 

1.91 

0.91 

1.45 

-3.25 

-2.05 

-1.40 

0.66 

Q39B 

Last  EVAIVFITREP  was 
fair/accurate 

66.7 

2.22 

1.12 

4.20 

-1.53 

-1.00 

-0.58 

0.63 

Q39C 

Last  EVAIVFITREP  was 
conducted  in  a  timely 

manner 

53.3 

2.12 

1.03 

2.73 

-1.90 

-1.27 

-0.79 

0.69 

Q39D 

Able  to  submit  input  at  my  last 
EVAIVFITREP 

45.7 

2.03 

1.02 

2.18 

-2.12 

-1.45 

-0.91 

0.55 

Q39E 

Last  promotion 
recommendation  was  fair 

64.9 

2.14 

1.08 

3.86 

-1.64 

-1.15 

-0.59 

0.54 

Q39F 

Satisfied  with  present 
EVAIVFITREP  system 

56.3 

2.66 

1.18 

1.79 

-1.75 

-0.90 

-0.15 

1.45 

Q39G 

Most  qualified  and  deserving 
Sailors  score  highest  on  the 
EVALs/FITREPs 

45.5 

3.11 

1.25 

1.21 

-1.58 

-0.39 

0.54 

2.29 

5  i 


t 

o 


4.65 


CL 


Strongly 

disagree 


Disagree  Neither  agree  Agree  Strongly  agree 
nor  disagree 


Categories  of  Mean  Performance  Evaluation-Short  Form  Scores 


Figure  15.  Mean  Performance  Evaluation/  Fitness  Report  scale  scores  by 

mean  short  form  scores. 

Recognition 

The  Recognition  scale  contains  only  two  items:  item  40A  (adequately  recognized  for 
accomplishments  on  EVALs/FITREPs)  and  item  40B  (Adequately  recognized  for 
accomplishments  with  awards).  Both  items  have  similar  psychometric  properties  (see 
Table  16).  Item  40A  was  selected  as  a  l-item  short  form  because  it  focuses  on  general 
recognition  rather  than  specifically  on  awards.  As  shown  in  Figure  16,  mean  Recognition 
scale  scores  increase  steadily  with  increasing  levels  of  agreement  on  item  40A. 
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Table  16 

Recognition  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q40A 

Adequately  recognized  for 
accomplishments  on  EVALs/ 
FITREPs 

81.6 

2.44 

1.17 

3.31 

-1.66 

-0.89 

-0.35 

0.89 

Q40B 

Adequately  recognized  for 
accomplishments  with 
awards 

83.7 

2.76 

1.24 

3.08 

-1.36 

-0.64 

-0.04 

1.16 

Item  40A  (I  feel  that  I  have  been  adequately  recognized  for  my 
accomplishments  on  my  EVALs/FITREPs) 


Figure  16.  Mean  Recognition  scale  scores  by  1-item  recognition  measure. 


Career  Progression 

Earlier  analyses  of  the  2005  NPS  data  suggested  that  the  NPS  scale  on  Career 
Development  should  be  split  into  two  scales,  measuring  Career  Progression  and  Career 
Counseling/Guidance.  The  psychometric  properties  of  the  two  items  on  the  Career 
Progression  scale  are  presented  in  Table  17.  Both  items  have  similar  properties  and 
could  be  suitable  one-item  measures  of  Career  Progression.  Item  41A  (clearly  defined 
path  for  my  designator,  rating,  or  community)  was  selected  because  it  accounts  for  a 
higher  percentage  of  variance  (78%)  in  the  long  form  scores.  This  relationship  is 
presented  graphically  in  Figure  17. 
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Table  17 

Career  Progression  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q4 1A 

Clearly  defined  path  for  my 
designator,  rating,  or 
community 

78.3 

2.40 

1.10 

2.14 

-2.02 

-1.21 

-0.49 

1.09 

Q41B 

Sufficient  progress  in  my 
advancement 

73.9 

2.29 

1.01 

2.94 

-2.09 

-1.34 

-0.62 

0.85 

Strongly  Disagree  Neither  agree  Agree  Strongly  agree 

disagree  nor  disagree 


Item  41A  (I  have  a  clearly  defined  career  path  for  my  designator,  rating,  or 

community) 


Figure  17.  Mean  Career  Progression  scale  scores  by  1-item  career 

progression  measure. 

Career  Counseling/  Guidance 

The  means,  standard  deviations,  percentage  of  scale  variance,  and  IRT  parameters 
for  the  two  Career  Counseling/Guidance  items  are  displayed  in  Table  18.  The  items 
generally  have  similar  values  for  each  of  these  measures.  Item  41D  was  selected  as  the  1- 
item  Career  Counseling/Guidance  short  form  given  its  higher  IRT  slope  parameter  (a  = 
5.56).  The  strong,  positive  relationship  between  item  41D  and  the  long  form  is  presented 
graphically  in  Figure  18. 
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Table  18 

Career  Counseling/  Guidance 


I RT  Parameters 


# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q41C 

Given  adequate  counseling/ 
guidance  by  my  immediate 
supervisor 

89.7 

2.76 

1.16 

3.80 

-1.44 

-0.66 

0.04 

1.28 

Q41D 

Given  adequate  counseling/ 
guidance  by  my  career 
counselor 

89.2 

2.84 

1.18 

5.56 

-1.24 

-0.59 

0.14 

1.26 

Item  41 D  (I  have  been  given  adequate  counseling/guidance  on  my  career 
development  by  my  division,  department,  or  command  career  counselor) 


Figure  18.  Mean  Career  Counseling/  Guidance  scale  scores  by  1-item  career 

counseling/  guidance  measure. 


Detailing 

Item-level  descriptive  statistics  for  the  six  items  on  the  Detailing  scale  are  presented 
in  Table  18.  The  overall  satisfaction  item,  item  24F  (satisfied  with  my  detailer),  was 
chosen  as  the  l-item  Detailing  short  form  because  it  accounts  for  the  greatest  variance 
and  shows  a  strong  relationship  with  Detailing  long  form  scores  and  has  the  largest 
slope  parameter  (see  Table  19  and  Figure  19). 
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Table  19 
Detailing  scale 


Item 

% 

1 RT  Parameters 

# 

Description 

Variance 

Mean 

SD 

a 

bl 

b2 

b3 

b4 

Q42A 

Satisfied  with  detailing  process 

68.9 

2.83 

1.10 

2.67 

-1.48 

-0.70 

0. 13 

1.75 

Q42B 

Clear  understanding  of  detailing 
process 

44.6 

2.60 

1.09 

1.47 

-2.33 

-1.04 

-0.15 

1.75 

Q42C 

Detailer  responds  in  a  timely 

manner 

73.4 

2.62 

1.07 

3.33 

-1.54 

-0.93 

0.05 

1.23 

Q42D 

Detailer  is  an  advocate  for  my 
needs/ desires 

81.3 

2.90 

1.09 

5.34 

-1.23 

-0.65 

0.35 

1.42 

Q42E 

Detailer  is  receptive  to  resolving 
conflicts 

81.6 

2.80 

1.03 

5.71 

-1.33 

-0.79 

0.28 

1.44 

Q42F 

Satisfied  with  my  detailer 

85.1 

2.71 

1.06 

7.39 

-1.30 

-0.86 

0.13 

1.24 

Item  42F  (I  am  satisfied  with  my  detailer) 


Figure  19.  Mean  Detailing  scale  scores  by  1-item  detailing  measure. 


Final  Short  Forms 

Table  20  presents  the  final  short  forms  of  the  19  individual  scales.  The 
percentage  of  variance  in  long  form  scores  accounted  for  by  the  short  forms 
ranges  from  70  percent  for  the  Fairness  scale  to  90  percent  for  the  Immediate 
Supervisor  scale. 
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Table  20 

Short  Forms  of  scales  on  the  Navy-wide  Personnel  Survey 


Item 

%  of  Variance 

#  Description 

in  Long  Form 

Mean 

SD 

Availability  of  Resources 

8A  Adequate  qualified  personnel 

8C  Adequate  spare  parts  and/or  supplies 

84% 

3.62 

0.90 

Morale 

84% 

3.26 

0.74 

10B  Performance  evaluation  system 
IOC  Supply  of  spare  parts/supplies 

10D  Quality  of  Navy  training  programs 

10H  Command  leadership 

10J  Workload 


Gender  1  ntegration 

72% 

2.21  0.96 

Women  are  being  successfully 

12C  integrated 

Workplace  Climate 

80% 

3.81  0.93 

Amount  of  responsibility  I  have  in  my 
13B  job 


Opportunity  for  personal  growth  and 
13D  development  on  the  job 


TEMPO 

19A  Time  spent  at  permanent  duty  station 

72% 

2.22 

1.08 

1  mpact  on  Personal  Life 

21A  Career  gets  in  the  way  of  personal  life 

79% 

2.89 

1.17 

1  mmediate  Supervisor 

23F  Satisfied  with  immediate  supervisor 

90% 

2.14 

1.14 

Command  Leadership 

24F  Satisfied  with  command  leadership 

89% 

2.29 

1.10 

Communication 

82% 

3.55 

0.83 

Navy  clearly  communicates  goals  and 
25A  strategies 


Command  leadership  keeps  me 
25D  informed  of  Navy  policies 


J  ob  Security 

26C  Future  in  Navy  appears  secure 

78% 

2.39 

1.12 

Fairness 

Confident  policies  affecting  size  of  Navy 
will  be  administered  fairly  and 

27D  consistently 

70% 

2.92 

1.06 
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Table  20 

Short  Forms  of  scales  on  the  Navy-wide  Personnel  Survey 


Item 

#  Description 

Navy  1  mage 

1  would  recommend  the  Navy  as  a  good 
28C  place  to  work 

%  of  Variance 
in  Long  Form 

76% 

Mean 

2.40 

SD 

1.08 

Organizational  Commitment 

1  feel  a  strong  sense  of  belonging  in  the 
37E  Navy 

84% 

2.46 

1.17 

Advancement/  Promotion 

Satisfied  with  advancement/promotion 

38B  system 

77% 

2.98 

1.26 

Performance  Evaluations/  Fitness  Reports 

39B  Last  EVAL/FITREP  was  fair/accurate 
Satisfied  with  present  EVAL/FITREP 

39F  system 

81% 

3.56 

1.00 

Recognition 

Adequately  recognized  for 

40A  accomplishments  on  EVALs/FITREPs 

82% 

2.44 

1.17 

Career  Progression 

Clearly  defined  path  for  my  designator, 
41A  rating,  or  community 

78% 

2.40 

1.10 

Career  Counseling/  Guidance 

Given  adequate  counseling/guidance  by 
41D  my  career  counselor 

89% 

2.84 

1.18 

Detailing 

42F  Satisfied  with  my  detailer 

85% 

2.71 

1.06 

Note:  Scores  for  short  forms  containing  more  than  one  item  are  computed  as  the  mean  of  items 
comprising  the  scale. 


Navy  Climate  I  ndex- Short  Form  (NCI  -SF) 


As  described  in  an  earlier  report  (Bann  et  al.,  2006)  and  shown  in  Figure  20, 
overall  Navy  Climate  was  conceptualized  as  including  seven  underlying 
constructs  measured  by  the  NPS:  (1)  workplace  climate,  (2)  organizational 
commitment,  (3)  morale,  (4)  job  security,  (5)  communication,  (6)  fairness,  and 
(7)  Navy  image.  The  long  form  of  the  Navy  Climate  Index  was  developed  by 
combining  the  NPS  scales  measuring  each  of  these  constructs  to  capture  both  the 
current  view  of  climate  and  perceptions  of  the  future  that  may  impact  it.  In  this 
chapter  development  of  a  short  form  of  the  NCI  (i.e.,  NCI-SF)  is  described  using 
the  newly  developed  short  forms  of  the  scales. 
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Figure  20.  Constructs  comprising  Navy  Climate  I  ndex. 


Development  of  the  Navy  Climate  I  ndex- Short  Form 

To  develop  the  NCI-SF,  a  confirmatory  factor  model  was  conducted  using  the 
short  forms  of  the  scales  corresponding  to  each  of  the  constructs  underlying  Navy 
Climate.  Figure  21  presents  a  path  diagram  for  the  second-order  confirmatory 
factor  model.  In  this  model,  researchers  tested  whether  the  seven  short  form 
scales  (Workplace  Climate,  Organizational  Commitment,  Morale,  Job  Security, 
Communication,  Fairness,  and  Navy  Image)  form  a  single  construct  representing 
overall  Navy  Climate.  As  with  the  development  of  the  long  form  of  the  NCI, 
correlated  errors  were  permitted  between  two  pairs  of  scales:  (a)  Workplace 
Climate  and  Fairness,  and  (b)  Navy  Image  and  Organizational  Commitment. 
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The  fit  indices  suggest  that  the  model  had  a  good  fit,  indicating  that  the  seven 
scales  may  be  combined  into  an  overall  Navy  Climate  Index  (CFI  =  0.98,  TLI  = 
0.96,  SRMR  =  0.03).  Table  21  presents  the  factor  loadings.  The  scales  had  similar 
factor  loadings  with  values  ranging  from  0.56  for  Organizational  Commitment  to 
0.69  for  Workplace  Climate. 

The  factor  loadings  for  the  scales  were  then  used  to  compute  scores  for  the 
NCI-SF.  Weights  for  each  scale  were  computed  based  on  the  scale’s  factor  loading 
divided  by  the  sum  of  the  factor  loadings.  Specifically,  scores  for  the  Climate 
index  short  form  were  computed  using  the  formula  outlined  in  the  following 
worksheet  (see  Table  22).  Index  scores  may  range  from  o  to  too  with  higher 
values  indicating  a  more  positive  Navy  climate. 


Figure  21.  Path  diagram  for  confirmatory  factor  model  of  Navy  Climate 

I  ndex- Short  Form. 
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Table  21 

Results  of  second-order  confirmatory  factor  model  for  Navy 

Climate  I  ndex 


Standardized 


Path  Coefficient 

Navy  Climate  I  ndex 

Morale  0.67 

Workplace  Climate  0.69 

Communication  0.65 

Job  Security  0.58 

Fairness  0.62 

Navy  I  mage  0.67 

Organizational  Commitment  0.56 

Model  Fit  I  ndices 

Comparative  Fit  Index  0.98 

Tucker-Lewis  I  ndex  0.96 

Standardized  Root  Mean  Square  Residual  0.03 


Note:  Similar  to  the  long  form  of  the  Navy  Climate  Index,  correlated  errors  are  permitted 
between  the  following  two  pairs  of  scales:  (1)  Workplace  Climate  and  Fairness,  and  (2) 
Navy  Image  and  Organizational  Commitment. 
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Table  22 

Worksheet  for  calculating  Navy  Climate  I  ndex- Short  Form  scores 


Scale 

Short  Form 
Score 

Scoring 

Coefficient 

Scale  score  x 
scoring 
coefficient 

Row 

Workplace  Climate 

Mean  of  items 
13B  and  13D 

X 

0.15 

(1) 

Organizational 

Commitment 

Item  37E 

X 

0.12 

(2) 

Morale 

Mean  of  items 
10B-D,  10H,  and 
10J 

X 

0.15 

(3) 

J  ob  Security 

Item  26C 

X 

0.15 

(4) 

Communication 

Mean  of  items 
25A  and  25D 

X 

0.14 

(5) 

Fairness 

Item  27D 

X 

0.14 

(6) 

Navy  Image 

Item  28C 

X 

0.15 

(7) 

Sum  of  scale  scores  x  scoring  coefficients  (rows  1-7)  =  _  (8) 

Subtract  1  from  row  8  =  _  (9) 


Navy  Climate  Index-Short  Form  Score:  =  _  (10) 

Multiply  row  9  by  25 


Validation  of  Navy  Climate  I  ndex- Short  Form 

Next,  the  equivalence  of  the  short  and  long  forms  of  the  NCI  was  evaluated. 
The  correlation  between  the  two  forms  was  0.97  and  the  short  form  accounted  for 
94  percent  of  the  variance  in  the  long  form  scores,  suggesting  that  the  two  forms 
are  highly  related.  To  further  explore  the  equivalence  of  the  two  forms,  the  NCI- 
SF  scores  among  various  demographic  groups  were  compared  to  determine  if  the 
patterns  of  differences  are  similar  to  those  found  with  the  long  form.  As  with  the 
long  form  of  the  NCI,  significantly  higher  scores  on  the  NCI-SF  were  found  for 
those  who  are  male,  have  a  higher  education,  are  married,  have  children  under 
age  21  in  their  household,  are  an  officer  or  warrant  officer,  are  not  in  their  first 
term  of  service,  and  not  currently  deployed  (Table  23). 
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In  addition,  we  compared  mean  NCI-SF  scores  by  responses  to  the  following 
three  items,  measuring  tone  and  job  satisfaction: 

Q29A:  How  would  you  rate  Navy  tone? 

Q29B:  How  would  you  rate  your  current  command’s  tone? 

Q65:  Considering  everything,  how  satisfied  are  you  with  Navy  life? 

As  shown  in  Figures  22  to  24,  there  is  a  significant,  positive,  linear 
relationship  between  the  NCI-SF  and  Sailors’  perceptions  of  tone  and  their  job 
satisfaction  (p  <  .0001).  Sailors  reporting  high  tone  or  job  satisfaction  had  more 
positive  climate  scores,  as  measured  by  the  NCI-SF. 

Finally,  the  relationship  between  NCI-SF  scores  and  the  following  three  items 
measuring  retention  intentions  was  explored: 

Q36A:  I  plan  to  serve  out  my  current  term  of  service  or  obligation. 

Q36B:  I  plan  to  reenlist  or  continue  with  my  career  in  the  Navy  at  my  next 
decision  point. 

Q36C:  I  plan  to  stay  in  the  Navy  for  a  full  career  if  possible. 

All  three  items  were  significantly,  positively  related  to  NCI-SF  scores  (p  < 
.0001).  However,  as  found  with  the  long  form  of  the  NCI,  this  relationship 
appears  to  be  stronger  for  re-enlistment  plans  (Q36B)  and  plans  to  remain  in  the 
Navy  for  a  full  career  (Q36C)  than  plans  to  serve  out  the  current  term  of  service 
(Q36A). 
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Table  23 

Mean  Navy  Climate  I  ndex  scores  by  demographic  characteristics 


Characteristic 

Mean 

SD 

P 

Gender 

Male 

63.15 

16.96 

<  .0001 

Female 

59.28 

18.18 

Education 

High  school  or  less 

60.73 

18.14 

<  .0001 

Some  college  or  2-year  degree 

61.64 

17.68 

Bachelor's  degree  or  more 

64.37 

16.05 

Marital  Status 

Married 

64.52 

16.70 

<  .0001 

Not  married 

59.18 

17.61 

Children  under  21  living  in 
household 

Yes 

64.01 

16.85 

<  .0001 

No 

60.79 

17.53 

Paygrade 

Enlisted 

61.26 

17.93 

<  .0001 

Warrant  Officer 

66.99 

13.96 

Officer 

64.68 

15.65 

First  enlistment  or  term  of 
service  in  Navy 

Yes 

57.90 

18.03 

<  .0001 

No 

65.44 

16.01 

Currently  on  deployment 

Yes 

59.23 

19.16 

.0016 

No 

62.69 

17.06 
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Q29A:  How  would  you  rate  Navy  tone? 


Figure  22.  Mean  Navy  Climate  I  ndex-Short  Form  scores  by  respondent 

ratings  of  Navy  tone. 
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Q29B:  How  would  you  rate  your  current  command's  tone? 


Figure  23.  Mean  Navy  Climate  I  ndex-Short  Form  scores  by  respondent 

ratings  of  command's  tone. 
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Very  Dissatisfied  Neither  Satisfied  Very  satisfied 
dissatisfied  satisfied  nor 

dissatisfied 


Q65:  Considering  everything,  how  satisfied  are  you  with  Navy  life? 


Figure  24.  Mean  Navy  Climate  Index-Short  Form  scores  by  respondent 
ratings  of  satisfaction  with  Navy  life. 
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Q36A:  I  plan  to  serve  out  my  current  term  of  service  or  obligation 

Figure  25.  Mean  Navy  Climate  Index-Short  Form  scores  by  retention 
intentions:  Plans  to  serve  out  current  term  of  service  or  obligation. 
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Q36B:  I  plan  to  reenlist  or  continue  my  career  with  the  Navy  at  my 
next  decision  point 


Figure  26.  Mean  Navy  Climate  I  ndex- Short  Form  scores  by  retention 
intentions:  Plans  to  reenlist  or  continue  career  with  Navy  at  next 

decision  point. 
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Figure  27.  Mean  Navy  Climate  I  ndex- Short  Form  scores  by  retention 
intentions:  Plans  to  stay  in  Navy  for  a  full  career. 
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Conclusion  and  Discussion 


This  report  outlines  the  exploration  of  potential  short  forms  for  the  19  scales 
on  the  2005  Navy-wide  Personnel  Survey.  The  primary  goal  was  to  develop  1- 
item  measures  for  each  of  the  scales  which  could  be  administered  when  time 
and/or  respondent  burden  is  of  particular  concern.  Using  both  statistical  and 
content-based  criteria,  14  l-item  short  forms  were  successfully  developed.  Short 
forms  for  four  of  the  remaining  scales  contained  two  items  each  and  the  short 
form  for  the  final  scale  contained  five  items,  primarily  due  to  concerns  about 
ensuring  content  coverage.  Each  of  the  short  forms  accounted  for  at  least  70 
percent  of  the  variance  in  the  long  form  scores  and  contained  items  with  high 
levels  of  discrimination,  no  floor  or  ceiling  effects,  and  appropriate  content. 

In  addition  to  developing  short  forms  for  each  of  the  individual  scales,  a  short 
form  for  the  Navy  Climate  Index  was  also  developed,  referred  to  as  the  NCI-SF. 
Confirmatory  factor  analyses  supported  combining  the  short  forms  of  seven 
scales  (Workplace  Climate,  Organizational  Commitment,  Morale,  Job  Security, 
Communication,  Fairness,  and  Navy  Image)  into  a  single  NCI-SF  score.  A 
worksheet  was  developed  that  can  be  used  by  future  users  to  easily  compute 
scores  for  the  NCI-SF.  The  NCI-SF  demonstrated  similar  relationships  with  other 
variables  (i.e.,  demographics,  tone,  job  satisfaction,  and  retention  intentions)  as 
the  NCI  long  form,  supporting  its  validity.  Furthermore,  the  NCI-SF  was  strongly, 
positively  correlated  with  the  NCI  (r  =  0.97)  and  accounted  for  94  percent  of  the 
variance  in  the  NCI  scores. 

In  conclusion,  the  results  suggest  that  several  of  the  scales  on  the  NPS  may  be 
substantially  shortened,  in  some  cases,  with  minimal  loss  of  reliability.  In 
particular,  the  findings  suggest  that  reducing  the  length  of  the  Navy  Climate 
Index  by  73  percent  (i.e.,  removing  35  of  the  48  items  on  the  NCI)  produces 
nearly  identical  scores,  given  the  high  correlation  between  the  long  and  short 
forms  of  the  NCI.  The  new  13-item  NCI-SF  that  was  developed  could  be 
administered  as  a  part  of  a  Navy  Quick  Poll  to  provide  Navy  leadership  with  rapid 
feedback  concerning  Navy  climate.  Future  studies  could  explore  the  development 
of  appropriate  cut-points  or  benchmarks  for  the  NCI-SF  in  relation  to  important 
outcomes,  such  as  retention  intentions. 
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